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ABSTRACT 

This study evaluates the efficacy of the Lexia Reading 
(LR) programme with a cohort of 37 students in a 
Decile 1 primary school. The students were randomly 
assigned to experimental (who used LR) and control 
groups (who did not use LR). The WIAT-II was used to 
provide pre and post measures of literacy skills. Results 
indicated that students who used LR did not outperform 
students in the control group and no statistical 
significance was found between the two groups at 
post -test. The study discusses the evidence base that 
Lexia presents in support of LR and raises questions in 
relation to LR efficacy. The study concludes that the LR 
computer programme's evidence base and efficacy is 
equivocal. 
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INTRODUCTION 

Computers are now ubiquitous in schools and as a 
result there is an ever-growing number of computer 
software packages being marketed to schools. Some 
of these packages appear to offer effective solutions 
to some of the most intractable problems faced by 
teachers and many, such as Lexia, now target students 
who might be described as presenting with special 
needs. As such, these programmes may appear very 
attractive to schools. Packages are generally targeted 
at individual students underpinned by the rationale 
that the students will benefit from Computer Assisted 
Instruction (CAI). 

When using a computer, students can be presented 
with auditory, visual and tactile-kinaesthetic stimuli 
as they learn. Carlisle and Rice (2002) write that 
CAI was primarily developed as a way of increasing 
instructional time and practice for students without 
placing further demands on teacher time. They further 
note that CAI enables instruction to be tailored to 
individual students, can allow for self-paced learning, 
can provide immediate feedback to students and 


that CAI can be both fun and engaging for students, 
especially for those who have reading difficulties. 

One other possible 'benefit' for the student is that 
they are withdrawn from classroom work which may 
be of less interest to them than spending time on the 
computer. In this way CAI can be quite appealing to 
students. 

There has been extensive debate over the last 20 
years as to the effectiveness of CAL Slavin, Cheung, 
Groff and Lake (2008) conducted a review of the 
literature in relation to CAL Of the eight studies that 
were reviewed (involving a total of 12,984 students) 
the overall weighted mean of the effect size of CAI 
programmes was +0.10. This is a very weak positive 
correlation and is on the verge of being negligible 
(Coolican, 2007). These findings are in line with 
practice guidelines for reading instruction from the 
US Department of Education's Institute of Education 
Sciences, which found that while there was little 
evidence to show that CAI was effective, CAI was 
an area that could have some potentially positive 
effects for students who used it (US Department of 
Education, 2010). 

Lexia Reading is one of the main programmes 
marketed by Lexia Learning Systems (LLS). Lexia 
Reading (LR) is a computer-based supplementary 
reading tool aimed at improving reading skills. It has 
an age range of 4 years to adult (and can also be used 
for ESOL students). LR is designed to supplement and 
complement classroom instruction; however it can be 
used as a stand-alone tool (Lexia Learning Systems, 
2006). LR uses games and interactive activities to 
"build skills with explicit practice in phonemic 
awareness and phonics while promoting gains in 
vocabulary, fluency and comprehension" (Lexia 
Learning Systems 2010, p.2). 

As students begin LR they are placed at a level that 
matches their ability. In order to progress through 
the levels they are required to complete tasks, which 
they must achieve before being able to move on to a 

higher level. 
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Within the Lexia programme a multisensory approach 
is intended to engage learners with the material. 
Students use headphones to listen to words/word 
sounds, and then interact with the computer, using a 
mouse to click on images to provide responses to the 
questions related to the word/sound. Other exercises 
do not require the students to listen to sounds/words 
but rather to complete the task on the screen using 
the mouse or keyboard. 

Lexia offer three main sources of supporting 
information by way of providing an evidence base: 
the first is peer- reviewed journal articles and the 
second is testimonial studies largely written by RTLBs 
and schools. Lastly, the company offers a webinar. 

There are three articles, cited on the Lexia website, 
from peer-reviewed academic journals. In exploring 
this evidence it appears that these articles were 
all written following extensive work in the United 
States of America with a group of public schools in 
Revere, Massachusetts (Macaruso, Hook & McCabe 
2006; Macaruso & Rodman 2009; Macaruso & 
Walker 2008). It appears that Lexia was given a rare 
opportunity to tailor-make their programme to fit with 
the schools' curriculum. These three studies appear to 
support the value for students who accessed the Lexia 
programme, particularly those students who were 
furthest behind (described as 'at risk'). 

There are a number of studies which have been written 
by Resource Teachers: Learning and Behaviour (RTLB), 
schools and a single personal testimonial which Lexia 
currently place on their website. These studies present 
information which is highly supportive of the Lexia 
programme. 

Lexia also claim that the What Works Clearinghouse 
and the Florida Centre for Reading Research have 
positively evaluated the Lexia programmes. 

The research team was invited to attend a webinar 
hosted by LR (a combined telephone/internet tutorial 
and discussion). During the Webinar the research team 
asked what progress might be expected from students 
who used LR. The LR representative confirmed that 
we might expect to see one years progress in one term 
(most likely in 'at risk' students) for students who used 
LR. 

Schools with a high demand for literacy teaching 
could be considered as the target for the presentation 
of this positive evidence. In particular, schools may 
well feel that the amount of teacher-time they are 
able to direct to raising literacy is limited, and the 
promise of having the process automated to some 
extent can easily be seen as attractive. In particular, 
Lexia's summary of the research shows potential: 


• Lexia Reading Improves Reading Score in grades 
K - 3 and Middle School 

• Lowest Performing Students Benefit the Most from 
using Lexia Reading 

• Title 1 Students Using Lexia Reading Close the 
Achievement Gap (Title 1 Students is a term used 
in the USA. They are equivalent to NZ students 
functioning at Level 1 of the curriculum or below) 

• Benefits of Lexia Reading are Tied to Strong Use 
Patterns 

• Teachers Strongly Endorse Lexia Reading (Lexia 
International, 2011). 

Given the positive indications the research team 
adopted the following hypothesis: given a single term 
of prescribed learning with LR, students will show 
significant improvement in literacy scores (one year in 
one term), when compared with students in a control 
group. The null hypothesis is that despite access to 
the Lexia programme (for the experimental group) no 
significant differences between the two conditions will 
be found. 

METHOD 

In order to further inform Lexia's claims the research 
team sought to explore Lexia's efficacy with 'at risk' 
students in a New Zealand educational context. The 
team were invited to help support a Decile 1 primary 
school which was considering purchasing the Lexia 
Reading Programme. Lexia provides free of cost, an 
unlimited number of licences for the duration of a single 
term (10 weeks). 

Participants 

Forty students were identified, ten in each of four 
combined school year classes; Years 1 -2, Years 3-4, 
Years 4-5, Years 5-6. All the children were identified 
by the school as 'at risk' in terms of their literacy i.e. 
their literacy was tracking at two years or more below 
their chronological age. The ten students from each 
class were randomly assigned to the control or the 
experimental condition. Two learners with English as 
an additional language were excluded from the study 
and one learner left the school, leaving a total of 37 
participants. 

Procedure 

The students followed normal classroom programmes 
and curriculum with the exception that the 
experimental group took part in LR for at least 100 
minutes per week for a single school term in 2010. 
Some students had significantly more time than this. 


Weaving educational threads. Weaving educational practice. 
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Measures 

The pre and post tests for the study were selected from 
the Wechsler Individual Achievement Test, (WIAT- 
II, 2nd ed,). This test was selected as it is arguably 
comparable to the tests used in the three peer- 
reviewed studies on the LLS research website (Lexia 
International, 2011). Adjustments were made to allow 
for differences in the New Zealand and Australian 
education systems. For the administration of this test all 
five year old students were treated as pre-schoolers (as 
pre-schoolers, the 10 five year old children were only 
administered the Word Reading and Spelling subtests). 
The following tests and descriptions are taken directly 
from the WIAT-II manual: 

Word Reading - assesses early reading (phonological 
awareness), word recognition and decoding skills. 

Reading Comprehension - assesses the types of reading 
comprehension skills taught in the classroom or used 
in everyday life. 

Spelling - assesses the ability to write dictated letters, 
letter blends, and spell words. 

Pseudoword Decoding - assesses the ability to apply 
phonetic decoding skills to 'nonsense' words. 


Additionally, the team sought to evaluate curriculum 
data supplied by the school alongside the results of 
the WIAT-II pre and post tests. The researchers also 
devised a 'Student Voice' questionnaire to explore 
learners' self-perception and their enjoyment of the 
Lexia programme. Interviews with members of the 
teaching staff were also conducted. 

Independent sample f tests on the pre-test data 
between the control and experimental conditions 
revealed no significant differences across the four 
WIAT-II subtests prior to beginning the trial. 

Results 

All children achieved levels of Lexia usage that fell 
within the required range. Using the WIAT-II data, 
age equivalencies were calculated in months for 
the purposes of data analyses. The following tables 
show the means and standard deviations across the 
two conditions and the four subtests. The difference 
between the pre and post test means are included 
at the end of each table. Tables 1 and 2 record the 
control and experimental data respectively, and Table 
3 allows comparison of the control and experimental 
means across the four sub-tests. 


Table 1 

Mean scores and standard deviations across the control sub-tests (all values in months) 


Subtest 

Pre-test 

mean 

Standard 

Deviation 

Post-test 

mean 

Standard 

Deviation 

Difference 
between means 

Word Reading 

77.05 

14.84 

80.11 

15.73 

3.06 

Spelling 

78.35 

11.14 

75.53 

21.99 

-2.82 

Reading Comprehension 

86.22 

17.10 

83.67 

18.56 

-2.55 

Pseudoword decoding 

81.09 

9.48 

81.82 

8.46 

0.73 


Table 2 


Mean scores and standard deviations across the experimental sub-tests (all values in months) 


Subtest 

Pre-test 

mean 

Standard 

Deviation 

Post-test 

mean 

Standard 

Deviation 

Difference 
between means 

Word Reading 

79.2 

12.3 

76.2 

13.3 

-3 

Spelling 

74.5 

14.15 

78.8 

10.79 

4.3 

Reading Comprehension 

76.67 

5.32 

77.5 

5.21 

0.83 

Pseudoword decoding 

86.86 

35.98 

74.5 

6.74 

-12.36 


Table 3 

Comparison of means between the control and experimental conditions (all values in months) 


Subtest 

Control Pre-test 

mean 

Experimental 
Pre-test mean 

Control Post-test 

mean 

Experimental 
post-test mean 

Word Reading 

77.05 

79.2 

80.11 

76.2 

Spelling 

78.35 

74.5 

75.53 

78.8 

Reading Comprehension 

86.22 

76.67 

83.67 

77.5 

Pseudoword decoding 

81.09 

86.86 

81.82 

74.5 
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A student voice questionnaire was also administered 
as an additional information source. The results are 
included in Appendix A. 

The curriculum data supplied by the school was not 
complete at the end of the study and has therefore not 
been included in the results section. 

When the independent sample t-tests were repeated 
on the post-test data between the experimental and 
control conditions across the four WIAT-II subtests, 
no statistical significance was found. The results do 
not serve or support any significant advantage for 
the experimental group. The null hypothesis must 
therefore be accepted that despite access to the Lexia 
programme (for the experimental group) no significant 
differences between the two conditions were found. 

Discussion 

From a Lexia standpoint these results would appear 
to be very disappointing. The cohort of 37 'at risk' 
readers who completed the trial would arguably 
be Lexia's target group. These learners all worked 
on Lexia at or above the minimum amount of time 
required by the programme as stipulated by Lexia for 
a period of a school term, which is supported by the 
trial period offered by Lexia to schools. In searching 
for a rationale for the uninspiring results generated by 
this research it seems important to revisit the evidence 
base in a little more detail. 

The testimonial research, as it is presented on 
Lexia's website, is typically narrative rather than 
experimental. Whilst acknowledging the excellent 
intentions of these projects, the research team 
found that there is little or no evidence of rigorously 
applied methodologies, the use of control groups, 
pre and post measures or statistical analysis. The 
researchers therefore struggled to see how they might 
convincingly support the evidence-base for Lexia. 

In support of LR, the LLS website states "the What 
Works Clearinghouse (WWC) finds Lexia Reading 
to be effective, meeting the WWC research 
standards" (Lexia International, 2011). The WWC 
was established in 2002 by the US Department 
of Education's Institute of Education Sciences to 
provide professions with guides to the effectiveness 
of programmes, practice guidelines and policies 
concerning education, including literacy and 
numeracy. LLS claims that the Lexia Reading 
programme is one of only ten programmes that 
meets the evidence standards set out by the WWC 
and shows "positive or potentially positive effects 
in at least two of the four beginning reading skills 
(alphabetics, comprehension, fluency and general 
reading achievement)" and "Based on the studies 


reviewed by WWC, Lexia Reading was found to have 
potentially positive effects on alphabetics and reading 
comprehension, and showed statistically significant 
effects in general reading achievement for subgroups 
of at-risk students" (Lexia International, 2011). 

The researchers examined the What Works 
ClearingHouse report pertaining to Lexia Reading. 

LLS presented 11 studies to the WWC for evaluation. 
Of these 11 studies, only two met the evidence 
standards with one further study meeting the 
evidence standards with reservations (US Department 
of Education, 2009). The other eight studies did 
not meet the evidence standards due to flaws in 
research design, methodology or conclusions. 

The WWC report determined that the evidence 
in support of the Lexia Reading programme to be 
"small for alphabetics, fluency, comprehension, and 
general reading achievement" (US Department of 
Education, 2009). The WWC found that the Lexia 
Reading programme had "potentially positive effects 
on alphabetics, no discernable effect on fluency, 
potentially positive effects on comprehension, and no 
discernable effects on general reading achievement" 
(US Department of Education, 2009). This appears to 
be at odds with the statements made by LLS. 

On closer examination the team found the three 
Macaruso studies are far from unequivocal. Macaruso 
(2006) in the first of the trio of studies available, 
worked with 1 79 students from ten first grade classes, 
allocated to experimental and control conditions. 
Initially no significant difference was found 
between these two groups which was disappointing 
considering the advantages of matching Lexia 
directly to a literacy curriculum over a six month 
period. Macaruso et al., (2006) note this apparent 
failure and embark upon further analysis. These 
endeavours did reveal that there was apparently a 
significant advantage for those in the experimental 
group described as 'at risk' (also known as Title 1 
Students in the USA) when compared with the same 
group of students in the control group. However, 
this advantage was only found for 'letter sound 
correspondences' in contrast to 'recognising basic 
story words' where no significant advantage was 
found. 

The second study (Macaruso et al., 2008) examines 
Lexia in kindergartens. Encouragingly, the study 
claims to have shown significant differences between 
the experimental and control conditions. However, 
the research team noted that the pre and post test 
measures used were not the same. Instead, all 
children were tested on a reading test (the Gates 
McGinitie) by way of identifying differences only 
at the end of the Lexia programme. This proved 
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very difficult for the research team to overlook 
and arguably the team did not agree with the 
rationale shared in the study for this aspect of their 
methodology. 

It would seem important for the research team to 
acknowledge that testing is seldom perfect and the 
use by the researchers of a test standardised on an 
Australian population is not, arguably, an ideal method 
of measurement. However it has only been used within 
this study's specific experimental methodology. It has 
not been compared with other data gathered by the 
school or post hoc tests. Furthermore, the adjustment 
for ages discussed in the introduction provides some 
protection from the influence of age on literacy ability. 
The team aspired to include school curriculum data 
on reading development to further inform the study. 
However, this data was incomplete at the time the 
study finished and has therefore not been included. 

The third Macaruso study appears to build on Lexia's 
apparent benefit for a cohort of 47 sixth and seventh 
grade middle school students with identifiable special 
needs in the area of literacy. The participants are 
described as 'attending remedial reading classes'. 

In contrast to the previous study, Macaruso et ah, 
(2009) employs the Woodcock-Johnson III Tests 
for Achievement. This standardised test has seven 
subtests and has been standardised alongside the 
WIAT-II used in the present study. Through the use 
of statistical analysis, Macaruso et al. was only able 
to show a significant advantage for the experimental 
group for the Word Attack sub-test. No significant 
differences were found in any of the other six areas 
tested (letter-word identification, reading fluency, 
reading vocabulary, passage comprehension, oral 
comprehension and spelling). In making this claim 
the researchers noted that in the area of word attack 
the control group's mean scores pre to post dropped 
from 87.7 to 85.3. As the post-test score for the 
experimental group was 88.3 it is not difficult to see 
that the apparent reversal of the control group has 
helped to make the findings significant. The question 
of why the control group made negative progress in 
this area is not fully explored. 

A preference for simple statistical analysis might be 
seen by some as a potential weakness of the present 
study. In the Macaruso studies a number of tests and 
investigations were conducted with skill and rigour. 
However the present research group would argue 
that from the way in which Lexia is marketed the data 
gathered should be unequivocally in favour of those 
students in the experimental condition. In reading the 
discussion sections of the Macaruso articles it becomes 
clear that there are some very strong claims made in 
favour of the Lexia programme but these would not 


appear to be fully supported by the results. 

By way of support for schools, Askov and Bixler offer a 
list of criteria by which to evaluate the appropriateness 
of computer software for educational purposes (1998, 
cited in Dunsmuir & Clifford, 2003). This list asks how 
well any given programme matches the more general 
curriculum objectives. The learners in the Macaruso 
studies "were taught with Language! An intensive 
reading programme that includes exercises to improve 
phonic skills" (Macaruso & Rodman, 2009, p. 109). It 
may be that any advantages claimed by the Macaruso 
studies are as a result of the opportunity to match Lexia 
with the curriculum but even this argument appears to 
be insecure. 

In the most recent of the Macaruso studies (2009) it 
is noted that in order for students to make progress 
"further instruction and practice are necessary" 
(Torgesen et al., 2003, cited in Macaruso 2009, 
p.109). In the same study there is a very brief 
reference made to additional study support on page 
106: "Students ... branch to more highly scaffolded 
practice activities when necessary". This perhaps 
signals the use of additional one-to-one instruction 
which would potentially have a very significant 
influence on scores in the experimental group. Whilst 
almost an aside of the 2009 study, the researchers 
and the participating school found the use of one-to- 
one instruction to be a major issue. 

When a learner has more than three failures on a 
task on LR, a yellow dot is placed on the record 
for the teacher to see. These dots are an indication 
to the teacher that the student requires one-to-one 
instruction. The LR system carries hundreds of pages 
of one-to-one instruction exercises to this end. When 
the school agreed to the LR trial they were unaware 
of these yellow dots. At no time were the yellow dots 
mentioned to the school or the researchers by LLS. 
When they began to appear on the student records 
the teachers were surprised to hear that they were 
then required to spend considerable additional one- 
to-one instruction time to the students. The school 
made the decision to not deliver the additional 
teaching materials to the students due to the extensive 
time demands of such individualised teaching. 

It is important to note that all of the students had 
yellow dots on their records at some stage during 
the trial. When a student fails to make progress at LR 
a teacher is required to give the student one-to-one 
instruction in the specific skill until it is established. 
They are then able to go back on LR and pass the 
level they were stuck on. This pattern is repeated 
every time a student has difficulty with a skill-set in 
LR. It might be argued that LR was not delivered with 
fidelity during this study but the contrary argument 
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is that through the delivery of significant one-to-one 
tuition until the learner has successfully learnt the 
identified skill, a student cannot fail on LR. 

We identify what appears to be a self-fulfilling 
prophecy. It would seem to the researchers that the 
benefits of LR lie not necessarily in the computer 
instruction medium, but rather in that it support 
teachers to provide students with targeted one-to- 
one instruction in specific skill areas that are key 
to literacy development. This form of instruction 
(one-to-one ) is widely supported by evidence and 
research as being an effective way of supporting 
literacy development (Carlisle & Rice, 2002). LLS 
appear to support this notion in that these lessons 
are an integral part of supporting students to succeed 
on LR. It does seem important to note that a single 
yellow dot can require over 90 pages of colour 
photocopying alongside the teaching time to deliver 
these additional materials. 

As this work has been completed there has been a 
growing concern within the research team that the 
marketing and efficacy of school targeted software do 
not necessarily share the same agenda. The team have 
questioned the evidence base and, at best, the answers 
have been equivocal. 

It is an ubiquitous feature of conclusions to research 
articles that more detailed and exhaustive research 
needs to be undertaken and this case is no exception. 
This was, after all, a comparatively small study limited 
to a single school. The discovery of additional teaching 
materials requiring significant time and resources is a 
further variable which warrants further exploration. In 
the final analysis there seem to be two programmes 
at work for the learner: computer time and direct 
teaching time. The individual instruction ensures that 
any shortcomings of the computer programme are 
ameliorated. 

CAI has a glamorous profile in schools but if it does 
not work, is it worth the investment of scarce school 
funds? This study found that CAI (as in LR) did not 
seem to help struggling readers. A future larger study 
is now needed to verify and extend the present 
results. 
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APPENDIX A 


The Student Voice Questionnaire 

The student voice questionnaire contained seven questions. For all items, apart from Question 5, the participants 
were asked to indicate their answer on a scale ranging from 0-10, where 0 is the least and 10 is the most. The 
researchers followed a semi-structured set of instructions to make sure each participant understood how to offer an 
answer. Question 7 was only for those in the experimental group and asked how much they enjoyed being part of 
the programme. The mean result for Question 7 was 8.13 (with a standard deviation of 3.67) suggesting that, despite 
scores across a wide range, participants reported that they enjoyed working on the Lexia programme. Question 5 
asked participants to list their strategies to reading an unfamiliar or difficult word. The table below (Table 4) presents 
the means across the two conditions for all items.Table 5 records the verbatim responses of the participants to 
Question 5. 


Table 4 

Questionnaire questions 1, 2, 3, 4, -, 6 and 7 mean scores for across conditions 


Question 

Pre-test mean 

control 

Post-test mean 
control 

Pre-test mean 
experimental 

Post test mean 
experimental 

How much do you like reading at home? 

7.63 

7.88 

7.05 

8.16 

Like reading at school? 

7.05 

7.4 

5.76 

5.55 

How good are you at reading? 

7.3 

7.18 

5.84 

7.32 

How good does your teacher think you are 
at reading? 

7.85 

7.55 

7.47 

7.76 

How much do you enjoy school? 

9.23 

7.55 

8.68 

8.89 

How much did you enjoy Lexia? 

N/A 

N/A 

N/A 

8.13 


Table 5. 

Verbatim responses to Question 5. 


Control Group pre-test 

Control Group post-test 

sound it out 

sound it out 

get another book 

get a different book 

sound it out 

skip it 

ask for some help maybe 

sound it out 

sound it out 

sound it out or go back and read it again 

sound it out 

skip it or sound it out 

sound it out in my head 

chunking 

sad because what one 

you say 1 can't do it 

1 think and then 1 know 

ask a teacher 

1 ask teacher to help me 

ask the teacher to help me read it 

happy or sad 

my teacher tells me what word it is 

no answer 

just look away just say it 

put book back 

1 like going to the park 1 like going to the beach 

just think 

sound it out 

look at picture and sound it out 

sound it out 

get a little angry and keep trying 

have a go 
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sound it out 

sound it out 

sound it out 

sound it out 

stop and sound it out 

sound it out 

skip it 

skip it and move on 

Experimental Group pre-test 

Experimental Group post-test 

don't know 

sound it out 

sound it out 

sound it out 

try, chunk it 

skip it 

try to think it out and if you don't know it you can skip 
it and come back to it 

sound it out 

don't know (shrug) 

sound it out 

chunk it up 

chunk it 

my dad helps me when 1 don't know it. He tells the 
girls to help you 

ask the teacher 

don't know 

ask the teacher 

ask my mum 

ask the teacher 

1 don't like reading at home so 1 just like reading at 
school 

1 get sad when my mum always don't give us food 

ask mum 

ask a teacher 

say the word in front of the whole word (pointed at 
first letter of word) 

1 sound the first word out 

no answer 

fun 

1 don't know 

go to my teacher 

miss it 

sound it out 

sound it out 

sound it out 

sound it out 

sound it out with your arm 

sound it out 

chunk it up 

sound it out 

read on 


Weaving educational threads. Weaving educational practice. 
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